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ABSTRACT 


System of Detection and Recognition of Sign Boards on Roads. This can be used to minimize the accidents count due to improper 
knowledge of signs on roads. it helps the drivers to guid the routeand also warn the driver about upcomeing dangers. This system 
inform in advance before the any danger occures. Purpose of this praper is to detect and recognize the signs and text on sign boards. 
Image capture though camera is pre-processed then sign board is detected and recognize the informatiom it contain. 
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INTRODUCTION 

Technology can play a major help tool in decreasing the count of 
road accidents. In general it is difficult for drivers to watch and 
get information about signs or text in sign board. In which driver 
can miss some of the warnings which can cause deaths. 
Sometimes due to road conditions or in nights, driver cannot 
read the sign boards like bump ahead or narrow bridge. 


This Traffic Sign detection and Recognition sytem can be a good 
tool to help the road users by providing vaild information about 
the sign or text that are present on the sign boards, by this System 
driver get information in advance about the upcomeing dangers 
which can make driver to prepare for it and overcome the danger 
and drive safely. This System uses computer vision and pattern 
recognition to find signs and symbols in automatic detection and 
recognition. 


MATERIALS AND METHODS: 
Detection and Recognition of Traffic sign system works in two 
stages. 
a. Detection- In this stage, Traffic sign presence or 
absence is detected. 


b. Recognition- In this stage, If the Taffic Sign detected 
then, detected traffic sign is recognised and classified. 


A important aspect is the need of a fast and reliable detection 
stage to boost-up the overall computation process. The detection 
stage serves as a primary step, enabling the system to identify 
and extract relevant knowledge from the input images. These 
extracted knowledge are subsequently forwarded as input to the 
recognition stage, where Convolutonal neural networks are 
leveraged for classification purposes. By employing 
Convolutional neural networks, the system can effectively 
categorize the detected traffic signs based on the different 
physical properties inherent to each type. 


To enhance the efficiency and robustness of the classification 
process, a strategic approach involves useing three distinct 
databases for training purposes. This segmentation of databases 
allows for the training of the Convolutional neural networks to 
be specific to the unique attributes of each type of traffic sign. As 


a result, the classification process becomes more streamlined, 
resilient, and expeditious, thereby contributing to the overall 
speed and accuracy of the real-time traffic sign Detection and 
Recognition system. 


The main steps included are: 

1. Image capturing from camera. 

2: Image enhancement for different lighting and whether 
conditions. 
Selecting candidate objects based on colour. 
Filtering the candidate objects based on shape. 
Recognising the filtered object using neural network 
Give visual image of the recognised sign and audio for 
the type of category of the traffic sign. 


DN 


A. Image Capturing from Camera 

Images are capturing useing the camera mounted on movie car. 
In this case, it is common for the captured images to contains 
blurriness, as well as variations in brightness, contrast, and 
clarity, which can become factors to effect image such as the 
time of day, location, and changeing weather conditions. 
Additionally, the presence of noise, stemming from climatic 
conditions, further complicates the detection process. 
Consequently, it becomes imperative to implement suitable pre- 
processing techniques to mitigate the impact of these diverse 
conditions on the images, thereby facilitating improved 
detection outcomes. 


To address these challenges, a two-fold pre-processing approach 
is adopted. Firstly, the application of a Gaussian noise filter use 
in the reduction of noise, enhancing the overall quality of the 
captured images. Additionally, the Weiner filter is used further 
eliminate noise and improve the overall clarity of the images. 
These filtering techniques collectively contribute to minimizing 
the adverse effects of noise and other irregularities present in the 
images. 


The pre-processed images are subsequently forwarded to the 
detection stage, ensuring that the input data for traffic sign 
recognition is optimized for improved accuracy and efficiency 
despite the challenging environmental conditions encountered 
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during image capture. 


B. Sign Detection Stage 

In this traffic sign detection based on color and shape analysis, an 
starting step involves the conversion of the captured image from 
the RGB color space to the YCbCr color space. The YCbCr color 
space is commonly used in digital video processing, facilitating 
the separation of luminance information into a single component 
(Y) and the representation of chrominance information as two 
color difference components (Cb and Cr). Specifically, Cb 
signifies the disparity between the blue component and a 
predefined reference value, while Cr represents the discrepancy 
between the red component and a designated reference value. 


The conversion between the RGB and YCbCr color spaces can 
be represented using the following formulas: 


For conversion from RGB to YCbCr: 
Y=0.299 R+0.587 G+0.114 B 
Cb=0.564 (B-Y) 

Cr=0.713 (R-Y) 


These formulas elucidate the computation process involved in 
transforming the RGB values of each pixel to their 
corresponding YCbCr components. By performing this 
conversion, the luminance and chrominance information can be 
effectively separated, enabling a more focused analysis of the 
color and shape attributes essential for the detection of traffic 


signs. 


1)Colour Based Analysis: 

To facilitate the classification of traffic signs into two groups, 
namely red traffic signs and blue traffic signs, a important step 
involves the extraction of colors, given that colors serve as a key 
distinguishing feature of these signs. Consequently, a color 
segmentation process is implemented within the YCbCr color 
space to effectively isolate the target color regions within the 
image. 


Following the color segmentation, a thresholding technique is 
applied to facilitate the distinct separation of red and blue color 
objects present in the image. The outcome of this step is the 
generation of a binary image, wherein the identified regions 
corresponding to red and blue colors are represented as 
foreground pixels, while the remaining regions are denoted as 
background pixels. 


Furthermore, to refine the segmented regions and ensure the 
cohesiveness of the identified color objects, morphological 
operations are employed. Specifically, the application of 
morphological closing operations aids in the elimination of any 
isolated or stray pixels present within the segmented regions. 
This process contributes to the creation of a more defined and 
accurate representation of the red and blue color objects within 
the traffic sign images, thereby facilitating the subsequent stages 
of analysis and detection. 


2) Shape Based Analysis: 

In the process of traffic sign analysis, while color segmentation 
aids in the initial selection of candidate objects, subsequent 
filtering of these candidates is executed through shape detection. 
This involves the following steps: 


Labeling Connected Regions: Connected candidate pixels are 
identified and grouped using the 8-neighbor connectivity 
approach, allowing for the formation of distinct candidate 
objects. As multiple candidate objects might be present, an 
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efficient filtering mechanism is essential to retain potential 
traffic signs while eliminating irrelevant elements. 


Leveraging Shape Characteristics: Shape detection serves as a 
pivotal filtering criterion, where four primary shapes—triangle 
(indicative of alert signs), circle (representative of compulsory 
signs), rectangle (symbolizing informational signs), and 
octagon (exclusively denoting STOP signs)—are utilized for 
classification purposes. 


Template Matching Using Similarity Measures: The detection 
process hinges on the comparison of the binary image obtained 
from the color-segmented road sign with the predefined objects 
stored in the template database. To facilitate this comparison, the 
sample and segmented images must possess identical 
dimensions or undergo resizing. The evaluation of similarity is 
performed through the utilization of distance measures, such as 


the Euclidean distance, for cross-correlation in template 
matching. 


Classification of Detected Signs: \f the similarity measure 
surpasses a predetermined threshold value, the analyzed object 
is identified as a traffic sign and subsequently classified into one 
of the designated classes. 


By implementing this systematic approach, the traffic sign 
detection system can effectively discern and categorize various 
traffic signs, contributing to enhanced road safety and efficient 
traffic management. 


C. Sign Recognition/ Classification Stage 

Upon successful identification of the presence of a traffic sign. 
The transmission of the processed frame to the recognition and 
classification stage for further analysis. 


1) Pictogram thresholding: 
After the confirmation of the presence ofa traffic sign within the 
frame, the recognition process focuses on isolating and 
extracting the pictogram contained within the traffic sign. This 
entails the implementation of a thresholding operation, which 
involves the following steps: 


Pixel-Wise Analysis: The frame obtained from the detection 
stage is subjected to a pixel-by-pixel analysis. Each pixel's 
intensity is compared against a predefined threshold value. 


Thresholding: Pixels with intensity values exceeding the set 
threshold are converted to white pixels, whereas pixels with 
intensity values below the threshold are converted to black 
pixels. This process aids in segregating and highlighting the 
pictogram within the traffic sign, effectively distinguishing it 
from the background. 


Stray Pixel Removal using Morphological Operations: 
Following the thresholding operation, any residual stray pixels 
are eliminated using morphological operations. Specifically, 
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morphological operations such as erosion and dilation are 
employed to refine the extracted pictogram, ensuring the 
removal of any extraneous or insignificant pixel clusters. 


By executing these operations, the system can accurately and 
efficiently extract the pictogram embedded within the traffic 
sign, facilitating the subsequent stages of analysis and 
interpretation essential for precise traffic sign recognition. 


2) Classification using neural network: 

One kind of deep learning algorithm that works especially well 
for tasks involving picture recognition and processing is the 
convolutional neural network (CNN). Convolutional, pooling, 
and fully connected layers are some of the layers that make it up. 


Convolutional Layer: 

Ina convolutional neural network (CNN), a type of layer known 
as a convolutional layer applies a collection of filters to the input 
data in order to produce feature maps that highlight the presence 
of features that have been discovered in the input. After each 
convolution operation, a CNN applies Rectified Linear 
Unit(ReLU) activation function transformation to the feature 
map, introducing nonlinearity to the model. The ReLU 
activation function is differentiable at all points except at zero. 
For values greater than zero, we just consider the max of the 
function. 


Pooling: 

Pooling is just reducing the size of the image without losing the 
features that we found with convolution. For example, a 
MaxPooling method will take in a shape of a matrix and return 
the larger value in that range. By doing this we can compress the 
image without losing the important features of this image 


Flattening: 
Flattening is nothing but converting a 3D or 2D matrix into a 1D 
input for the model this will be our last step to process the image. 


Fully connected layer: 
It takes the input from the previous layer and computes the final 
classification or regression task. 


Output Layer: 

The output from the fully connected layers is then fed into a 
logistic function for classification tasks like sigmoid or softmax 
which converts the output of each class into the probability score 
of each class 
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RESULTS: 

This process involves taking vedio as input automatically from 
the camera. Using cv2 library the video is converted into 
collection of frames and those frames will be input as images 
.These image as input, performing preprocessing to standardize 
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dimensions. Intial detect the Traffic sign and extracting features 
using a pre-trained Convolutional Neural Network (CNN) in 


recognition phase. which will generate the textual description. 
————— : 
= 


(e.g:-Output as turn Right) 


CONCLUSIONS: (Optional) 

This paper outlines a real-time method for the detection and 
recognition of Indian traffic signs, organized into three distinct 
classes based on their shapes. The system operates through two 
fundamental steps: initial detection of the traffic sign board, 
followed by a recognition process. To achieve this, the system 
employs a pattern matching technique to facilitate the 
comparison of different traffic signs. 


One of the key features of this system is its high-speed 
performance and robustness. It achieves this by excluding 
images containing signs with shapes diverging from the standard 
basic shapes at the initial stage. This selective process ensures 
that only relevant images are subjected to further analysis, 
thereby enhancing the system's efficiency and accuracy. 


To enhance the system's performance even further, future 
improvements could focus on fortifying its resilience against 
varying weather conditions and accommodating tilted signs. 
Implementing modifications to address these challenges would 
contribute to the system's adaptability and reliability, enabling it 
to operate effectively in diverse environmental conditions and 
thereby ensuring improved road safety and traffic management. 
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